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The  Honorable  John  Glenn 
Chairman,  Committee  on 
Governmental  Affairs 
United  States  Senate 

Dear  Mr.  Chairman: 

In  June  199 1 ,  we  testified  before  your  Committee  that  the  Internal  Revenue 
Service’s  (IRS)  ability  to  direct  and  control  its  procurement  activities  is 
critically  important  to  the  success  of  its  Tax  Systems  Modernization  (TSM) 
program.1  We  also  indicated  that,  so  far,  IRS  has  lacked  this  ability  and 
some  of  its  recent  procurements  have  rim  into  trouble. 

This  report  responds  to  your  request  that  we  review  one  of  these  troubled 
procurements-the  Treasury  Multi-User  Acquisition  Contract  (TMAC).  This 
contract  was  awarded  to  AT&T  in  July  1991  for  $  1.4  billion,  but  was  later 
successfully  protested  by  two  vendors— IBM  and  Lockheed.  These  protests 
have  added  to  the  delays  already  encountered  in  conducting  this 
procurement. 

You  asked  us  to  determine  what  IRS  (1)  did  to  cause  the  protests,  (2)  could 
have  done  to  avoid  the  successful  protests,  and  (3)  needs  to  do  to  prevent 
similar  successful  protests  of  future  TSM  procurements.  Our  objectives, 
scope,  and  methodology  are  detailed  in  appendix  I. 


Results  in  Brief 


The  TMAC  procurement  was  successfully  protested  because  IRS  did  not 
adequately  justify  awarding  this  contract  to  AT&T.  Although  AT&T’s 
$1.4  billion  proposal  was  found  to  be  technically  superior,  the  other 
vendors’  proposals  were  priced  between  $500  million  and  $700  million 
lower  than  AT&T’s.  IRS  failed  to  demonstrate  that  it  is  worth  paying  over 
one-half  billion  dollars  more  for  this  technical  superiority. 

To  avoid  the  successful  protests  by  IBM  and  Lockheed,  IRS  should  have 
done  a  better  price/technical  tradeoff  analysis.  IRS’  Source  Evaluation 
Board,  which  oversaw  the  TMAC  acquisition  process  and  recommended  that 
AT&T  get  the  award,  did  not  do  an  adequate  tradeoff  analysis  to  determine 
whether  any  offer  was  worth  its  cost  in  relation  to  the  other  offers. 
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Moreover,  IRS  missed  two  opportunities  to  ensure  a  better  analysis.  First,  it 
did  not  give  any  specific  guidance  to  the  Source  Evaluation  Board  on  the 
methodologies  and  criteria  to  be  used  in  analyzing  proposals  where 
(1)  technical  excellence  is  an  important  evaluation  factor  and  (2)  award  is 
made  to  other  than  the  lowest  bidder.  Specific  guidelines  would  have  given 
the  Source  Evaluation  Board  a  better  idea  of  how  to  do  an  acceptable 
analysis.  Second,  IRS  did  not  revise  its  analysis  after  it  received  advance 
review  comments  from  the  Department  of  the  Treasury  questioning  the 
validity  of  its  price/technical  tradeoff  analysis. 

To  avoid  similar  successful  protests  on  future  TSM  procurements,  IRS  needs 
to  include  specific  methodologies  and  criteria  for  conducting 
price/technical  tradeoff  analyses  in  its  procurement  guidelines.  These 
guidelines  need  to  be  made  an  integral  part  of  planning  documents 
associated  with  all  TSM  procurements. 


Background 


During  1990,  IRS’  computer  systems  for  administering  income  taxes 
accounted  for  over  $1.1  trillion  in  revenue  to  the  Treasury  and  processed 
over  200  million  taxpayer  returns  along  with  one  billion  other  documents, 
such  as  annual  wage  and  tax  statements  (Forms  W-2).  However,  these 
systems  use  outmoded  1950s  technology.  As  a  result,  they  are  inefficient, 
costly,  unresponsive,  and  unnecessarily  burdensome  on  taxpayers.  IRS  is 
making  its  third  attempt  in  25  years  to  modernize  these  systems.  This 
attempt  is  known  as  TSM.  According  to  IRS,  TSM  will  cost  about  $8  billion 
through  2008.  Because  of  earlier  failures  at  modernization  and  the  poor 
condition  of  IRS’  antiquated  systems,  the  IRS  Commissioner  has  stated  that 
TSM  cannot  be  postponed. 

Procurement  plays  a  vital  role  in  the  TSM  program.  However,  as  we  pointed 
out  in  our  June  25,  1991,  testimony,  IRS  has  lacked  the  ability  in  the  past  to 
properly  direct  and  control  its  procurement  activities.  We  noted  that  both 
internal  and  external  reviews  have  criticized  IRS  procurement  efforts  and 
IRS  was  reporting  its  procurement  system  as  a  material  internal  control 
weakness  under  the  Federal  Managers’  Financial  Integrity  Act. 

TMAC  is  a  1-year  contract,  with  annual  renewal  options  for  up  to  6  years,  to 
provide  up  to  3,200  minicomputers,  50,000  work  stations,  printers, 
networking  hardware  and  software,  an  integrated  office  automation 
system,  and  computer  maintenance  and  support.  The  contract  is  critically 
important  to  the  TSM  program;  most  of  the  items  under  this  contract  will  be 
used  for  certain  TSM  projects,  some  of  which  are  already  underway. 
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The  TMAC  procurement  process  took  much  longer  than  expected.  When  IRS 
released  the  request  for  proposals  in  January  1989,  the  agency  anticipated 
that  contract  award  would  occur  in  January  1990.  The  award  date  was 
delayed  primarily  because  of  the  time  IRS  needed  to  respond  to  vendors’ 
questions,  most  of  which  involved  clarification  of  IRS’  requirements.  In 
addition,  according  to  IRS  officials,  the  award  date  was  also  delayed  due  to 
the  time  necessary  to  obtain  proof  that  hardware,  software  and 
communication  items  contained  in  vendors’  proposals  were  commercially 
available.  By  the  time  IRS’  Source  Evaluation  Board  convened  in  May  1991 
to  begin  evaluating  the  final  proposals,  the  agency  was  already  16  months 
late  in  awarding  the  contract.2 

In  recommending  that  AT&T  be  awarded  the  contract,  the  Source 
Evaluation  Board  drew  upon  reports  prepared  by  a  technical  evaluation 
panel  and  a  business  management  evaluation  panel,  as  well  as  its  own 
analysis  of  the  competing  proposals.3  IRS  adopted  the  Source  Evaluation 
Board’s  recommendation  and,  on  July  15,  1991,  awarded  the  contract  to 
AT&T. 


IRS  Did  Not  Adequately 
Justify  Its  Selection  of 
AT&T 


The  TMAC  procurement  was  successfully  protested  because  IRS  failed  to 
adequately  justify  choosing  AT&T.  The  April  1991  technical  evaluation 
panel  report  showed  that  AT&T  offered  the  outstanding  technical  solution. 
Because  the  price  difference  was  so  great  between  AT&T’s  and  the  other 
vendors’  proposals,  the  Source  Evaluation  Board  prepared  a 
price/technical  tradeoff  analysis  to  determine  which  proposal  offered  the 
best  value  to  the  government.  The  Board  considered  five  value  categories 
(flexibility,  productivity,  availability,  training,  and  staffing)  and  found  that 
the  AT&T  offer  was  superior  in  all  five.  The  Board  therefore  concluded  that 
AT&T  should  be  awarded  TMAC. 


Lockheed  and  IBM  protested  the  award,  alleging  that  IRS  (1)  made  an 
irrational  price/technical  tradeoff  analysis,  (2)  did  not  follow  the  evaluation 
scheme  set  forth  in  the  request  for  proposals,  and  (3)  misevaluated 
protesters’  proposals.  The  General  Services  Administration  Board  of 
Contract  Appeals  (GSBCA),  in  its  September  25,  1991,  decision,  agreed  that 


2The  Source  Evaluation  Board  consisted  of  the  following  IRS  officials:  the  Chief  Information  Officer; 
Assistant  Commissioner  (Information  Systems  Management);  Director,  Office  of  Examination 
Programs;  and  Assistant  Director,  Contracts  and  Acquisition  Division. 

3The  technical  evaluation  panel  was  responsible  for  developing  the  statement  of  work  and  evaluation 
criteria,  and  for  reviewing  technical  proposals.  The  business  management  evaluation  panel  reviewed 
business  aspects  of  the  request  for  proposals’  statement  of  work,  and  offerors'  proposals. 
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the  analysis  was  flawed  because  it  did  not  address  the  price  differential, 
and  therefore  the  analysis  “had  no  value.”  Specifically,  the  GSBCA  stated 
that  the  price/technical  tradeoff  analysis  did  not  measure  the  benefits  the 
government  would  receive  from  the  different  offerors;  distorted  the 
evaluation  criteria  by  placing  more  emphasis  on  technical  features  than 
allowed  by  the  request  for  proposals;  and,  as  a  result,  did  not  measure 
whether  any  offer  was  worth  its  cost  in  relation  to  the  others. 

The  GSBCA  then  directed  IRS  to  (1)  prepare  a  suitable  price/technical 
tradeoff  analysis  that  would  comply  with  the  request  for  proposals  and 
either  confirm  the  previous  award  or  make  a  new  selection,  or  (2)  amend 
the  request  for  proposals  to  provide  a  clear  statement  of  its  intention  to 
emphasize  technical  features  over  cost  to  the  degree  it  believed  necessary. 

On  October  25,  1991 ,  in  response  to  IRS’  request  for  reconsideration, 
GSBCA  sustained  its  decision.  IRS  has  since  hired  a  contractor  to  provide 
assistance  and  guidance  in  preparing  another  price/technical  tradeoff 
analysis.  A  new  Source  Evaluation  Board  then  will  make  its  award 
recommendation.4  IRS  hopes  to  complete  the  analysis  and  make  a  final 
decision  by  March  31,  1992. 


IRS  Could  Have 
Prepared  a  Better 
Price/Technical 
Tradeoff  Analysis 


IRS  could  have  prepared  a  better  price/technical  tradeoff  analysis  of  the 
vendors’  proposals,  thereby  potentially  avoiding  the  successful  protests. 
However,  IRS  missed  two  opportunities  to  ensure  a  better  analysis. 

First,  IRS  contracting  officials  responsible  for  the  request  for  proposals  and 
source  selection  plan  did  not  give  the  Source  Evaluation  Board  any  specific 
guidance  on  what  methodologies  and  criteria  to  use  in  analyzing  large  price 
differences  among  the  offers.  For  example,  the  request  for  proposals 
contained  evaluation  criteria  which  stated  that  the  contract  would  go  to  the 
offeror  whose  proposal  contained  the  combination  of  technical, 
management  and  support,  and  cost  features  that  offered  the  best  value  to 
the  government.  It  cautioned  that  the  government  would  not  make  an 
award  at  a  significantly  higher  price  to  achieve  slightly  superior  technical 
features.  The  solicitation  also  noted  that  differences  between  technical 
features  and  overall  cost  to  the  government  would  be  compared,  but  did 
not  specify  how  this  would  be  done.  Technical  features  and  management 


4The  new  Source  Evaluation  Board  consists  of  the  TSM  Program  Manager;  Assistant  Commissioner 
(Information  Systems  Management);  Director,  Office  of  Information  Resources  Management, 
Department  of  Treasury;  and  Assistant  Commissioner  (Procurement). 
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and  support  features  were  each  assigned  100  evaluation  points;  no  points 
were  assigned  for  cost  features. 

Also,  IRS  source  selection  plans  are  intended  to  help  analyze  vendor 
proposals  by  providing  guidance,  such  as  the  types  of  evaluations  needed 
and  the  methodologies  and  techniques  to  be  followed.  However,  the  source 
selection  plan  for  the  TMAC  procurement  did  not  state  what  methodologies 
or  criteria  were  to  be  used  in  comparing  technical,  management  and 
support,  and  cost  features. 

Therefore,  neither  the  request  for  proposals  nor  the  source  selection  plan 
provided  the  Source  Evaluation  Board  with  any  methodology  for 
comparing  costs  with  technical  features.  Instead,  the  Board  was  left  to  its 
own  judgment  in  making  this  comparison.  Source  Evaluation  Board 
members,  working  with  the  information  they  received  from  the  technical 
evaluation  panel  and  business  management  evaluation  panel  that  strongly 
suggested  AT&T’s  technical  superiority,  decided  they  would  need  to 
determine  whether  AT&T’s  technical  superiority  was  worth  paying  over 
one-half  billion  dollars  more.  In  the  absence  of  specific  guidance,  they 
relied  on  their  experiences  with  other  procurement  analyses.  They 
ultimately  did  an  analysis  they  believed  was  adequate.  However,  IRS 
officials  told  us  that  considering  GSBCA’s  decision,  better  guidance, 
including  appropriate  methodologies  and  criteria,  might  have  enabled  them 
to  (1)  avoid  successful  protests  by  having  a  better  price/technical  tradeoff 
analysis  and  (2)  recognize  that  additional  guidelines  would  be  helpful  in 
conducting  future  price/technical  tradeoff  analyses. 

Second,  IRS  did  not  improve  the  price/technical  tradeoff  analysis  as 
recommended  by  the  Department  of  Treasury.  Because  of  the  size  of  the 
TMAC  procurement,  Treasury  regulations  required  IRS  to  obtain  Treasury 
review  and  concurrence  before  awarding  the  contract.  In  written 
comments  to  IRS  dated  July  9,  1991 ,  Treasury  stated  that  while  the  AT&T 
offer  resulted  in  the  superior  technical  proposal  in  most  areas,  IRS  needed 
to  reaffirm  that  the  award  to  AT&T  provided  the  best  value  to  the 
government.  Specifically,  Treasury  requested  stronger  analysis  to  support 
the  award. 

Although  Treasury’s  comments  stated  that  the  deficiencies  it  noted  should 
be  corrected  as  a  condition  of  Treasury’s  concurrence,  it  nevertheless  went 
ahead  and  concurred  with  the  award.  Treasury  did  not  require  a  written 
response  on  how  IRS  corrected  or  planned  to  correct  these  deficiencies. 
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Since  Treasury  concurred  with  the  award,  IRS  made  no  changes  to  the 
Source  Evaluation  Board’s  price/technical  tradeoff  analysis.  In  a  letter  sent 
to  Treasury  after  contract  award,  IRS  stated  that  it  considered  the  analysis 
used  to  achieve  best  value  as  being  appropriate.  IRS  offered  no  substantive 
arguments  to  support  this  position. 

Treasury  procurement  officials  recognize  that  they  need  to  be  more 
specific  in  telling  IRS  and  other  Treasury  bureaus  exactly  what  is  required 
in  responding  to  Treasury  comments.  Accordingly,  they  have  drafted 
revised  procurement  instructions  clarifying  the  required  action  to  be  taken 
with  respect  to  review  comments.  For  example,  the  draft  instructions 
require  each  Treasury  comment  to  be  labeled  so  that  contracting  officials 
will  clearly  know  which  corrections  are  mandatory  and  which  are 
discretionary. 


Conclusions 


The  tmac  procurement  shows  that  IRS  needs  to  improve  how  it  directs  and 
controls  its  procurement  activities.  Because  IRS  did  not  show  that  it  had 
made  a  proper  price/technical  tradeoff,  the  award  was  successfully 
protested.  As  a  result,  a  procurement  that  was  already  behind  schedule  has 
been  delayed  again.  IRS  now  is  retracing  its  steps  and  preparing  a  more 
convincing  price/technical  tradeoff  analysis  on  which  to  base  its  award  of 
this  contract. 

More  important,  however,  is  the  impact  of  this  delay  on  the  $8  billion  TSM 
program.  TMAC  is  one  of  several  procurements  that  are  critical  to  TSM. 
Without  TMAC,  IRS  will  lack  the  minicomputers,  work  stations,  and  other 
hardware  and  software  items  needed  to  move  forward  with  the  program. 
Until  IRS  significantly  improves  its  ability  to  direct  and  control  its  future 
procurement  activities,  TSM  will  be  vulnerable  to  further  delays. 


Recommendation  to 
the  Commissioner  of 
Internal  Revenue 


To  help  avoid  similar  successful  protests  of  future  TSM  procurements,  we 
recommend  that  the  Commissioner  of  Internal  Revenue  develop 
guidelines— including  appropriate  methodologies  and  criteria— to  be  used 
in  making  price/technical  tradeoff  analyses,  and  require  that  these 
guidelines  be  made  an  integral  part  of  the  planning  documents  for  all  such 
procurements. 
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Agency  Comments  and 
Our  Evaluation 


On  February  3,  1992,  IRS  provided  us  with  written  comments  on  a  draft  of 
this  report.  IRS  agreed  with  our  recommendation  that  guidelines  should  be 
developed  for  future  use  in  making  price/technical  tradeoff  analyses.  The 
agency  also  stated  that  it  expects  the  price/technical  tradeoff  analysis 
currently  being  conducted  for  the  TMAC  procurement  may  well  serve  as  a 
model  for  developing  these  guidelines.  These  actions  should  help  prevent 
successful  protests  of  this  kind  in  the  future. 

Although  IRS  agreed  with  our  recommendation,  it  did  not  agree  with  our 
finding  that  the  agency  could  have  prepared  a  better  price/technical 
tradeoff  analysis.  IRS  contends  that  there  are  no  specific  federal  guidelines 
for  agencies  to  follow  in  preparing  such  an  analysis,  and  that  in  the 
absence  of  such  guidelines  it  followed  what  it  believes  was  a  reasonable 
approach  in  preparing  its  analysis.  However,  we  agree  with  GSBCA  that  IRS’ 
analysis  was  inadequate,  and  believe  the  absence  of  specific  guidelines 
should  not  have  precluded  IRS  from  preparing  its  own.  The  size  of  the 
planned  TMAC  procurement  and  the  emphasis  on  technical  merits  should 
have  alerted  the  agency  to  the  fact  that  such  guidelines  may  be  needed. 
Detailed  IRS  comments  and  our  evaluation  are  contained  in  appendix  II. 


We  performed  our  review  between  October  1991  and  February  1992,  in 
accordance  with  generally  accepted  government  auditing  standards.  As 
agreed  with  your  office,  unless  you  publicly  announce  the  contents  of  this 
report  earlier,  we  plan  no  further  distribution  until  30  days  from  the  date  of 
this  letter.  We  will  then  send  copies  to  the  Secretary  of  the  Treasury;  the 
Commissioner  of  Internal  Revenue;  the  Director,  Office  of  Management 
and  Budget;  and  interested  congressional  committees.  Copies  will  also  be 
made  available  to  others  upon  request.  This  report  was  prepared  under  the 
direction  of  Howard  G.  Rhile,  Director,  General  Government  Information 
Systems,  who  can  be  reached  at  (202)  336-6418.  Other  major  contributors 
are  listed  in  appendix  III. 

Sincerely  yours, 


Assistant  Comptroller  General 
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Appendix  I  _ _ _ _ _ 

Objectives,  Scope,  and  Methodology 


In  response  to  a  request  by  the  Chairman,  Senate  Committee  on 
Governmental  Affairs,  we  reviewed  IRS’  Treasury  Multi-User  Acquisition 
Contract  (TMAC)  procurement  to  determine  what  IRS  (1)  did  to  cause  the 
protests,  (2)  could  have  done  to  avoid  the  successful  protests,  and 
(3)  needs  to  do  to  prevent  similar  successful  protests  of  future  TSM 
procurements.  We  reviewed  IRS,  Treasury,  and  General  Services 
Administration  Board  of  Contract  Appeals  (GSBCA)  documentation, 
including  evaluation  criteria,  procedures  and  plans  used  to  evaluate 
proposals,  agency  review  procedures,  review  comments,  minutes  of 
meetings,  and  correspondence.  We  also  reviewed  the  protest  file,  known  as 
the  “Rule  4  file.”  In  addition,  we  interviewed  IRS  and  Treasury  officials  to 
obtain  information  on  the  evaluation  and  review  process  of  the  TMAC 
procurement. 

We  performed  our  work  at  the  Department  of  the  Treasury,  General 
Services  Administration,  and  IRS  headquarters  in  Washington,  D.C.,  as  well 
as  IRS  offices  in  Falls  Church,  Virginia. 
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Comments  From  the  Internal  Revenue  Service 


Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


DEPARTMENT  OP  THE  TREASURY 
INTERNAL  REVENUE  SERVICE 
WASHINGTON,  D.C.  20224 


DEPUTY  COMMISSIONER 


Mr.  Ralph  V.  Carlone 
Assistant  Comptroller  General 
United  States  General  Accounting  Office 
441  G  Street,  N.W. 

IMTEC — Techworld,  Rm.  10028 
Washington,  DC  20548 

Dear  Mr.  Carlone: 

Thank  you  for  the  opportunity  to  review  the  draft  report 
entitled  "Tax  Systems  Modernization:  IRS  Could  Have  Avoided 
Successful  Protests  of  Major  Computer  Procurement"  (IMTEC-92-27) . 
While  we  do  not  fully  agree  with  GAO's  findings,  we  agree  with 
the  report's  recommendation  to  develop  guidelines  to  be  used  in 
making  price/technical  tradeoff  analyses  that  will  be  included  as 
part  of  the  planning  documents  for  procurements.  Our  comments  on 
other  aspects  of  the  report  are  enclosed. 

While  we  agree  that  additional  guidelines  on  conducting 
price/technical  tradeoff  analyses  are  needed,  we  have  not  found 
any  agency  that  has  issued  specific  guidelines  on  conducting 
price/technical  tradeoff  analyses.  We  expect  our  "lessons 
learned"  from  the  Treasury  Multi-User  Acquisition  Contract  (TMAC ) 
will  result  in  the  issuance  of  guidelines  for  conducting  future 
tradeoff  analyses  in  a  successful  manner. 

We  agree  that  one  of  the  important  elements  for  completing 
Tax  Systems  Modernization  is  a  highly  effective  procurement 
organization.  For  example,  the  IRS  elevated  its  procurement 
program  from  a  division  level  to  the  level  of  an  Assistant 
Commissioner.  It  selected  experienced  procurement  professionals 
for  the  positions  of  Assistant  Commissioner  and  Deputy  Assistant 
Commissioner.  It  has  also  increased  its  procurement  staffing  and 
has  attracted  numerous  qualified  and  experienced  procurement 
professionals  from  other  agencies.  The  IRS  continues  to  improve 
its  procurement  organization  as  evidenced  by  our  twenty-one 
successfully  completed  competitive  contract  awards  for  federal 
information  processing  resources  in  excess  of  one  million  dollars 
since  October  1990.  We  look  for  more  improvements  in  the  future. 
We  appreciate  your  underscoring  the  importance  TMAC  holds  to  a 
successful  Tax  Systems  Modernization  program. 

Best  regards. 


Enclosure 


Sincerely 
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RESPONSE  TO  GAO  DRAFT  REPORT  ON 
"TAX  SYSTEMS  MODERNIZATION:  IRS  COULD  HAVE 
AVOIDED  SUCCESSFUL  PROTESTS  OF  MAJOR  COMPUTER  PROCUREMENT" 


The  subject  report  finds: 

1.  That  the  IRS  could  have  avoided  successful  protests  of 
the  Treasury  Multi-User  Acquisition  Contract  (TMAC)  if  an 
adequate  price/technical  tradeoff  analysis  had  been  conducted; 

2.  That  the  IRS  missed  two  opportunities  to  ensure  a  better 
analysis  in  that: 

a.  IRS  contracting  officials  failed  to  give  specific 
guidance  on  the  methodologies  and  criteria  to  use  in 
analyzing  large  price  differences  between  offerors/ 
and  / 

b.  IRS  did  not  act  on  a  Department  of  the  Treasury 
(Treasury)  recommendation  to  improve  the 
price/technical  tradeoff  analysis. 

3.  That  until  IRS  significantly  improves  its  ability  to 
direct  and  control  its  future  procurement  activities/  TSM  will  be 
at  risk  of  failure. 

The  IRS  response  to  the  subject  draft  report  addresses  the 
following  three  issues: 

A.  The  Conduct  of  the  Price/Technical  Tradeoff  Analysis 

The  IRS  disagrees  with  the  GAO's  findings.  At  the  center  of 
the  issue  on  TMAC  is  the  GSBCA’s  conclusion  that  the  IRS  did  not 
conduct  an  adequate  price/technical  tradeoff  analysis.  In  other 
words/  the  IRS  did  not  explain  to  the  GSBCA's  satisfaction  why 
the  technically  superior  AT&T  proposal  was  worth  the  price 
differential  between  it  and  the  other  offers. 

There  are  no  specific  guidelines  or  "cookbook"  within  the 
Federal  Acquisition  Regulation  (FAR)  or  Federal  Information 
Resources  Management  Regulations  (FIRMR)  for  agencies  to  follow 
on  how  to  conduct  a  price/technical  tradeoff  analysis.  Prior  to 
the  TMAC  decision/  General  Accounting  Office  (GAO)  and  General 
Services  Administration  Board  of  Contract  Appeal  (GSBCA)  case  law 
indicated  that  award  to  a  higher-priced/  higher-technically-rated 
offeror  is  proper  when  the  decision  is  reasonable  and  consistent 
with  the  evaluation  criteria.  On  TMAC/  the  IRS  made  a 
comprehensive  analysis  of  both  subjective  and  objective  factors 
that  constituted  a  price/technical  tradeoff.  As  part  of  the 
objective  factors/  IRS  conducted  two  separate  analyses/  termed 
"Method  A"  and  "Method  B "z  to  consider  the  price/technical 
tradeoff  using  two  different  sets  of  assumptions.  The  results  of 
the  analyses  were  reviewed  by  the  IRS'  Internal  Audit  Division 


Page  12 


GAO/IMTEC-92-27  IRS  Could  Have  Awarded  Contracts 


Appendix  II 

Comments  From  the  Internal  Revenue  Service 


See  comment  2. 


See  comment  3. 


-2- 


and  by  IRS'  General  Legal  Services  (GLS).  Also/  the  Department 
of  the  Treasury/  in  their  review  of  the  procurement/  concurred 
with  Method  B  as  an  acceptable  approach  and  raised  questions  with 
Method  A.  The  GSBCA  found  both  methods  to  be  inadequate  and  gave 
the  IRS  the  option  to  conduct  an  adequate  price/technical 
tradeoff  analysis. 

In  addition/  page  10  of  the  GAO  report  states/  "Source 
Evaluation  Board  members  contend  that  better  guidance/  including 
appropriate  methodologies  and  criteria/  could  have  enabled  them 
to  avoid  successful  protests  by  having  a  better  price/technical 
tradeoff  analysis."  Members  of  the  Source  Evaluation  Board  did 
not  make  this  statement  to  the  GAO  team.  Based  on  existing  case 
law  at  the  time  they  were  making  the  price/technical  tradeoff 
analysis/  Source  Evaluation  Board  members  believed  they  were 
proceeding  appropriately.  In  light  of  the  TMAC  decision/ 
however/  it  is  recognized  that  additional  guidelines  would  be 
helpful  to  demonstrate  to  others  that  a  sound  business  decision 
was  made  and  how  it  was  made. 

B.  The  Treasury  Review  Process  as  it  Related  to  TMAC 

At  the  time  TMAC  was  awarded/  Treasury  Procurement 
Regulations  required  that  all  contracts  estimated  to  exceed  $1 
million  be  reviewed  and  concurred  in  by  Treasury's  Office  of 
Procurement.  Moreover/  the  Treasury  Procurement  Regulations 
required  that  on  contracts  in  excess  of  SI  million/  the  Treasury 
Procurement  Executive  sign  on  the  face  page  of  the  contract  as  an 
indication  of  his  approval. 

The  proposed  contract  to  AT&T  was  submitted  to  the 
Treasury's  Office  of  Procurement  on  July  1/  1991.  The  contract 
was  reviewed  and  concurred  in  by  a  memorandum  signed  by  the 
Treasury  Procurement  Executive  on  July  9/  1991/  which  states 
that/  "We  concur  in  the  award  of  the  proposed  procurement  action/ 
only  after  the  deficiencies  noted  in  the  attached  comments  have 
been  corrected."  The  Procurement  Executive  did  not  require  the 
IRS  to  respond  to  the  comments  as  he  could  have  by  checking  the 
appropriate  box.  More  importantly/  he  also  signed  the  face  page 
of  the  contract  indicating  his  approval.  Having  received  the 
Treasury  comments/  the  IRS  delayed  making  the  award  until 
July  15/  1991/  in  order  to  ensure  that  the  comments  had  been 
satisfactorily  addressed.  The  IRS  reviewed  each  comment  and 
provided  a  written  response  to  Treasury  which  clarified  each 
issue.  This  response  is  documented  in  the  contract  file. 

As  expected/  during  the  GSBCA  protest  process/  the 
Department  of  the  Treasury  comments  were  thoroughly  discussed  and 
analyzed.  There  were  18  substantive  (not  administrative) 
comments  in  the  Treasury  report.  Of  these  18/  neither  the 
protesters  nor  the  GSBCA  raised  or  addressed  nine  of  the 
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comments.  Of  the  nine  that  were  raised  by  the  protesters*  the 
GSBCA  initially  disagreed  with  Treasury  and  the  protesters  on 
five  of  the  comments  and  agreed  with  Treasury  and  the  protestors 
on  four  of  the  comments.  Later*  however*  in  a  Motion  for 
Reconsideration*  the  GSBCA  corrected  itself  on  one  issue  thereby 
concurring  with  the  IRS.  In  summary*  of  the  nine  substantive 
Treasury  comments  raised  during  the  protest  process*  the  GSBCA 
agreed  with  the  IRS  on  six  of  the  issues  and  disagreed  with  IRS 
on  three  of  the  issues. 

Although  Treasury  agreed  with  the  IRS  that  the  Method  B 
approach  to  conducting  a  price/technical  tradeoff  analysis  was 
appropriate*  the  GSBCA  rejected  both  the  Method  A  and  Method  B 
approaches . 

Simply  put*  the  record  does  not  support  a  finding  that  if 
the  IRS  had  followed  the  Department  of  the  Treasury's  review 
findings*  successful  protests  could  have  been  avoided. 

C .  The  IRS  Procurement  Program 

The  IRS  recognizes  that  the  success  of  our  Tax  Systems 
Modernization  (TSM)  program  is  dependent  upon  many  critical 
factors  including  a  highly  effective  procurement  program.  The 
IRS  has  made  significant  improvements  in  its  procurement  program* 
as  evidenced  by  its  successes  achieved  since  October  1990  and  is 
continuing  to  make  substantial  improvements. 

For  example: 

1.  In  November  1990*  the  IRS  elevated  its  procurement 
program  from  a  division  level  to  the  level  of  an  Assistant 
Commissioner.  It  selected  experienced  procurement  professionals 
for  the  positions  of  Assistant  Commissioner  and  Deputy  Assistant 
Commissioner.  It  has  also  increased  its  procurement  staffing  and 
has  attracted  numerous  qualified  and  experienced  procurement 
professionals  from  other  agencies. 

2.  Since  October  1990*  the  IRS  has  successfully  awarded  21 
federal  information  processing  resources  contracts  in  excess  of 
$1  million.  Each  of  these  was  a  competitive  procurement.  Each 
followed  the  same  review  process  as  TMAC .  These  contract  awards 
involved  many  unsuccessful  offerors  who  were  effectively 
debriefed  and  chose  not  to  protest. 

3.  Since  October  1990*  the  IRS  has  realized  successes  on  a 
number  of  "major"  contract  awards.  For  example: 

a.  The  Integrated  Collection  Systems  Contract  was 
awarded  on  December  28*  1990  to  IBM  for  $325*000*000. 
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b.  The  Integrated  Systems  Contract  was  awarded  on 
December  11,  1991  to  TRW  for  5300,000,000. 

c.  Eight  contracts  were  competitively  awarded  for  the 
Department  of  the  Treasury  Telecommunication  Systems  during  the 
period  May  31,  1991  through  December  23,  1991  in  the  total  amount 
of'  $254,876,778. 

d.  The  Impact  Printing  System  Contract  was  awarded  to 
Control  Data  Corporation  on  February  5,  1991  in  the  amount  of 
514,061,957. 

4.  Since  October  of  1990,  TMAC  is  the  only  GSBCA  or  GAO 
protest  the  IRS  has  received,  against  a  federal  information 
processing  resources  contract  award  in  excess  of  51,000,000.  In 
the  TMAC  decision,  the  GSBCA  found  significant  findings  in  favor 
of  the  IRS.  In  fact,  the  only  issue  that  the  GSBCA  found  with 
the  TMAC  procurement  was  with  the  price/technical  tradeoff 
analysis.  The  GSBCA  did  not  overturn  the  award  but  did  return 
the  procurement  to  the  IRS  for  further  analysis.  The  IRS  is 
currently  conducting  this  price/technical  tradeoff  analysis.  A 
working  group  was  established  which  consists  of  IRS  personnel  as 
well  as  external  experts  to  conduct  this  analysis.  This  group 
will  report  their  findings  to  a  newly  formed  Source  Evaluation 
Board  by  March  1992. 

5.  On  November  15,  1991,  the  Treasury  Department  raised 
the  procurement  review  threshold  for  the  IRS  from  one  million  to 
three  million  dollars.  This  change  was  made  as  a  result  of  a 
Treasury  oversight  review  performed  at  the  IRS  which  recognized 
the  improvement  at  IRS  in  the  management  of  acquisitions  and  in 
the  quality  of  the  IRS  procurement  program. 

RECOMMENDATION 

The  subject  report  recommends  that  the  IRS  "develop 
guidelines — including  appropriate  methodologies  and  criteria — to 
be  used  in  making  price/technical  tradeoff  analyses,  and  require 
that  these  guidelines  be  made  an  integral  part  of  the  planning 
documents  for  all  such  procurements." 

COMMENT :  In  light  of  the  TMAC  decision,  we  agree  that 

additional  guidelines  would  be  helpful.  Our  review  of  the 
Federal  Acquisitions  Regulations  and  Federal  Information  Resource 
Management  Regulations,  our  contacts  with  other  agencies,  and  our 
discussions  with  GAO  indicate  to  us  that  specific  guidelines  do 
not  exist  either  in  regulations  or  in  procedures  within  the 
federal  sector.  We  expect  that  the  analysis  currently  being 
conducted  on  the  TMAC  procurement  may  well  serve  as  a  model  for 
developing  such  guidelines  for  future  procurements. 


Page  15 


GAO/IMTEC-92-27  IRS  Could  Have  Awarded  Contracts 


Appendix  II 

Comments  From  the  Internal  Revenue  Service 


See  comment  5. 


-5- 


GENERAL  COMMENTS 

We  would  like  to  note  that  the  TMAC  award  date  was  delayed 
due  to  several  factors#  some  of  which  were  beyond  IRS  control. 
Between  the  release  of  the  RFP  (1/89)  and  proposal  submission 
(9/89) #  the  IRS  gave  written  answers  to  approximately  If 000 
vendor  questions  regarding  such  subjects  as  standards  and  other 
parts  of  the  specifications. 

Between  the  receipt  of  proposals  (9/89)  and  request  for  Best 
and  Final  Offers  (3/91) #  the  IRS  resolved  in  writing  over  IrOOO 
discussion  items.  During  the  period  between  11/90  and  3/91/ 
approximately  two  months  were  devoted  to  obtaining  proof  that 
every  hardware#  software#  and  communication  item  of  all  offers# 
was  in  fact  commercially  available.  This  degree  of  validation 
was  not  originally  anticipated.  The  time  taken  for  these 
validations  and  evaluations  was  productive#  as  the  GSBCA  found  no 
fault  with  the  technical  validations/evaluations#  and  not  a 
single  technical  issue  prevailed  in  the  final  analysis  of  the 
protest. 

Footnotes  on  pages  5  and  8  of  the  report  are  incorrect 
regarding  the  composition  of  the  Source  Evaluation  Board.  When 
the  Board  convened  in  May  of  1991#  it  consisted  of  the  Chief 
Information  Officer;  the  Assistant  Commissioner  (Information 
Systems  Management);  the  Director#  Office  of  Examination 
Programs;  and  the  Assistant  Director#  Contracts  and  Acquisition 
Division.  The  new  Source  Evaluation  Board  consists  of  the  TSM 
Program  Manager;  the  Assistant  Commissioner  (Information  Systems 
Management);  the  Director#  Office  of  Information  Resources 
Management#  Department  of  Treasury;  and  the  Assistant 
Commissioner  (Procurement). 
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GAO  Comments 


The  following  are  GAO’s  comments  on  the  Internal  Revenue  Service’s  letter 
dated  February  3, 1992. 

1 .  The  absence  of  specific  guidelines  or  a  “cookbook”  for  agencies  to 
follow  on  how  to  conduct  a  price/technical  tradeoff  analysis  should  not 
have  precluded  IRS  from  preparing  its  own.  The  size  of  the  planned  TMAC 
procurement  and  the  emphasis  on  technical  merits  should  have  alerted  the 
agency  to  the  fact  that  such  guidelines  may  be  needed. 

We  disagree  with  IRS’  contention  that  Treasury  concurred  with  Method  B  of 
the  price/technical  tradeoff  analysis  as  an  acceptable  analytical  approach. 
The  Treasury  comments  indicate  that  although  Treasury  believed  this  part 
of  the  analysis  was  better  than  Method  A  in  attempting  to  establish  the 
value  of  AT&T’s  proposal,  it  still  had  shortcomings.  However,  as  we 
mention  in  Comment  3  below,  the  point  we  are  making  is  that  although 
parts  of  the  analysis  may  have  been  better  than  others,  sufficient  questions 
were  raised  by  Treasury  to  alert  IRS  that  further  actions  were  needed. 

2.  The  statement  was  based  on  interviews  with  the  following  IRS  officials: 
Chairman  of  the  Source  Evaluation  Board;  Assistant  Commissioner, 
Procurement;  and  the  TMAC  Program  Manager.  We  have  modified  the 
statement  to  clarify  its  source  and  to  indicate  that  it  was  made 
retrospectively,  considering  the  gsbca’s  ruling. 

3.  The  fact  that  GSBCA  did  not  agree  with  certain  Treasury  comments  is  not 
relevant.  Our  report  refers  to  the  basis  for  the  decision  that  the 
price/technical  tradeoff  analysis  was  inadequate.  That  GSBCA  found  the 
analysis  more  inadequate  than  Treasury  is  also  not  relevant.  The  point  we 
are  making  is  that  Treasury’s  review  comments  strongly  indicated  that  the 
analysis  was  inadequate  and  IRS  chose  not  to  change  it.  Treasury  signed  off 
on  the  contract  but  requested  that  IRS  readdress  the  technical/price 
tradeoff  analysis  to  ensure  that  concerns  were  adequately  resolved.  In  its 
response  to  Treasury,  IRS  simply  attempted  to  justify  the  analysis. 
According  to  Treasury  officials,  IRS’  response  was  not  adequate. 

4.  While  we  are  not  able  to  comment  on  the  “significant  improvements” 
cited  by  IRS  in  its  procurement  program,  we  note  that  failure  to  effectively 
direct  and  control  even  a  few  large  procurements  could  adversly  affect  TSM. 
Such  is  the  case  with  TMAC,  which  IRS  recognizes  is  important  to  TSM. 

5.  Delays  in  the  TMAC  award  date  due  to  vendor  questions  have  been 
addressed  in  the  report.  We  revised  the  report  to  recognize  that  the  award 
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was  also  delayed  due  to  the  time  necessary  to  obtain  proof  that  hardware, 
software,  and  communication  items  contained  in  vendors’  proposals  were 
commercially  available.  We  also  modified  the  footnotes  to  reflect  an 
updated  composition  of  the  Source  Evaluation  Board. 
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Gregory  P.  Carroll,  Evaluator-in-Charge 
Frank  J.  Philippi,  Assignment  Manager 


(5X0766) 
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